<html>
<head><meta charset="utf-8"><title>documenting Rust syntax · t-lang · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/index.html">t-lang</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html">documenting Rust syntax</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="214522243"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522243" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522243">(Oct 26 2020 at 00:47)</a>:</h4>
<p>I'd like to broach the topic of adding otherwise invalid syntax to the language to accommodate macros and DSLs. <br>
The AST-level syntax is becoming so far removed from what is semantically valid that it is becoming increasingly difficult to document and explain what is "valid Rust syntax". There is a large queue of such changes that were made last year that have not been documented, because they tend to significantly complicate the documentation. At a certain point, there is more text explaining the differences between the AST syntax and the accepted language, than there is in the grammar itself.</p>
<p>I'm not sure why so many changes to the syntax of the language are made without an RFC, without community input, or even without team approval. To me, the syntax of the language is very important, and changes to it should be made with great care. I feel like various macro DSLs are being created to take advantage of rustc's parser instead of writing their own parser against raw token streams. The language then forever contains these extensions, even if the users that motivated them are no longer around.</p>
<p>I realize my voice is unlikely to sway the language team from continuing this trend. I recognize that these changes makes certain macro DSLs much easier to write. I would appreciate if someone could at least help with developing a strategy of how to document "valid Rust". I'm reluctant to add two grammars to the reference, but it is getting to the point that I don't see an alternative. Unfortunately, I feel like this adds significantly to the complexity budget of the language, which I feel is already very complex.</p>



<a name="214522478"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522478" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522478">(Oct 26 2020 at 00:53)</a>:</h4>
<p>w.r.t <code>writing their own parser against raw token steams</code>:</p>



<a name="214522486"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522486" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Mario Carneiro <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522486">(Oct 26 2020 at 00:53)</a>:</h4>
<p>Do you have some examples of such changes? (I don't doubt you at all, but I haven't been watching this closely)</p>



<a name="214522489"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522489" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522489">(Oct 26 2020 at 00:53)</a>:</h4>
<p>This can only be done for bang macros, which take an arbitrary delimited tokenstream</p>



<a name="214522533"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522533" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522533">(Oct 26 2020 at 00:54)</a>:</h4>
<p>For attribute macros, the target must be a syntactically valid rust attribute target, which can be parsed by the built-in parser</p>



<a name="214522550"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522550" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522550">(Oct 26 2020 at 00:55)</a>:</h4>
<p>Unfortunately, the syntax of attributes doesn't allow for looser parsing</p>



<a name="214522612"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522612" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522612">(Oct 26 2020 at 00:56)</a>:</h4>
<p>E.g. we need to be able to parse </p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">fn</span> <span class="nf">foo</span><span class="p">()</span><span class="w"> </span>-&gt; <span class="kt">bool</span> <span class="p">{</span><span class="w"></span>
<span class="cp">#[attr]</span><span class="w"> </span><span class="k">fn</span> <span class="nf">inner_fn</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
<span class="p">{</span><span class="w"> </span><span class="kc">true</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>



<a name="214522613"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522613" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522613">(Oct 26 2020 at 00:56)</a>:</h4>
<p>Yea, I recognize that it accommodates attribute macros better, and that bang macros in some cases would be uglier. I realize that there are good motivations for the changes. But added together they have consequences in terms of the total complexity.</p>



<a name="214522618"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522618" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522618">(Oct 26 2020 at 00:56)</a>:</h4>
<p>I am reluctant to point at any specific changes, because I don't want to single anyone out.</p>



<a name="214522634"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522634" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522634">(Oct 26 2020 at 00:57)</a>:</h4>
<p>(deleted)</p>



<a name="214522635"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522635" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522635">(Oct 26 2020 at 00:57)</a>:</h4>
<p>(deleted)</p>



<a name="214522636"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522636" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522636">(Oct 26 2020 at 00:57)</a>:</h4>
<p>I largely agree with you</p>



<a name="214522683"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522683" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522683">(Oct 26 2020 at 00:58)</a>:</h4>
<p>Individually, I think each of the changes have been a good idea (e.g. allowing <code>unsafe</code> on modules)</p>



<a name="214522707"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522707" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522707">(Oct 26 2020 at 00:59)</a>:</h4>
<p>But taken together, they create a 'psuedo-rust' that can be quite different from normal rust</p>



<a name="214522713"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522713" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522713">(Oct 26 2020 at 00:59)</a>:</h4>
<p>(off topic: zulip sent my message three times earlier)</p>



<a name="214522898"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522898" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522898">(Oct 26 2020 at 01:03)</a>:</h4>
<p>FWIW, I don't think this is something we should do without bound.</p>



<a name="214522948"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214522948" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214522948">(Oct 26 2020 at 01:04)</a>:</h4>
<p>I think it's OK to do when there's a natural reason for it (e.g. attributes everywhere).</p>



<a name="214534088"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214534088" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214534088">(Oct 26 2020 at 05:15)</a>:</h4>
<p>And thank you for bringing it up.</p>



<a name="214534345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214534345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> isHavvy <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214534345">(Oct 26 2020 at 05:22)</a>:</h4>
<p>Can we color tokens (red?) in the grammar where they parse correctly but aren't semantically allowed? Or otherwise attach information to them showing they're not allowed semantically? Or are there changes that aren't simple tokens? It would be useful to have a list of these cases somewhere I think, although the reference wouldn't really fit. Perhaps the rustc-dev book as an appendix?</p>



<a name="214628682"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/213817-t-lang/topic/documenting%20Rust%20syntax/near/214628682" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/213817-t-lang/topic/documenting.20Rust.20syntax.html#214628682">(Oct 26 2020 at 20:28)</a>:</h4>
<p>There were two categories of syntactic additions:</p>
<ul>
<li>Those from the "item syntax unification" effort - the logic there was always similar to "if function items supports a curly-braced body in some parent context, then they should support it in all parent contexts", this is how we got bodies for functions in foreign blocks, for example. (This is how we got <code>$item:item</code> working in all parent contexts as well.)</li>
<li>Arbitrary additions "for proc macros" - "this is syntactically valid, but never semantically valid", this is how we got visibilities on enums, unsafe on modules, maybe something else. I'd personally reject proposals from this category, but it isn't my decision.</li>
</ul>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>